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ABSTRACT 

During the last few years f a number of economists 
ha^Te studied the effects of employing different teaching resources on 
the performance of economics students. Despite widespread interest in 
the impact of various inputs on the SEJOunt learned by students, 
little attentioTi has been devoted to the choice of output measure. 
This paper argues in favor of the use of a variety of measures in 
analyzing resource effectiveness. To illustrate the importance of 
this position, several resource allocation questions are considered: 
(1) Do experienced teachers do a better job of teaching economics 
than less experienced ones? (2) Are graduate students as effective at 
teaching economics as members of the faculty? (3) Does the hour at 
which an economics course is taught have any effect on student 
performance? The data to answer these questions were obtained from 
pre and post questionnaires administered to successive macroeccnomic 
principles classes at Florida State Oniversity. A total of 301 
students were included in the sample. (Author) 
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TEACHER EFFECTIVENESS AND STUDENT PERFORMANCE 
Howard P. Tuckman 

In this paper we evaluate the effects of using experienced 
teachers as compared to graduate instructors in the macroecono- 
mic principles course at Florida State University. As con- 
trasted to earlier studies these questions are explored using 
five different measures of classroom "output."(l)(9)<10)(13)(19). 
Our 3?e8ults suggest that graduate students do as well as the 
overall faculty on each of the output measures > although exper- 
ienced teachers raise student learning scores over the sample 
average . 
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Determining Acceptable Measures of Student Perfo3?mance 

An acceptable measure of student performcince is a necessary 
pr»erequisite for any study of teacher effectiveness. The prob- 
lem is that using a single measure may have the undesired ef- 
fect of causing an increase in '"risasured" rather than "desired" 
output-. (8) In reality several things are going on in the class- 
room and, ideally, we should like to capture the most important. 
For this reason, we have chosen to work with five different 
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measures. The first is the Test of Understanding in College 
Economics (TUCE) which is a nationally normed and widely accepted 
test of learning. Participation by economists in question se- 
lection, weighting of subject matter, and testing for validity 
and reliability gives the exam a high degree of respectability. 
Several studies confirm its ability to discriminate between 
"good" and "poor" students (14) as well as its freedom from 
political bias and capacity to capture critical thinking. (m)(22) 
Because TUCE takes a full class period to administer, we found 
it necessary to select a limited number of questions from the 
exam. The evidence suggests that this did not substantially ef- 
feet test reliability.^ 

A second measure of performance was obtained through the 
construction of a twenty question test of economic attitudes. 
Mann and Fusfeld (MF) define attitude sophistication (AS) in 
terms of a student *s ability to take a position on an economic 
issue which accords with that of a majority of economists • (15) 
Measures of the MF type pose several problems since they are not 
free of value judgments (22) nor do they take account of exist- 
ing theoretical divisions within the field. One might also 
question the ability of any exam to capture truth in one line. 
Nevertheless, it seems reasonable to assume that economic 



1. Previous studies found that the TUCE questions have a high 
degree of internal reliability. (27) Using the Kuder- 
Richardson Formula 20, we obtained a test statistic of about 
0.6 for the sample. 
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attitudes are changed by a principles course and that the change 
is likely to be in the direction suggested by both the textbook 
and the lecturer • The measure presented in this paper repre- 
sents one way to capture attitude change. Obviously, others are 
also possible. 

Final grade of the student represents a third measure of 
performance. Because of the institutional arrangement at 
Florida State a common grading policy is not followed. This 
raises several problems in interpreting the grade variable : 
(1) Instructors differ in what they think is important. Thus, 
the same student may get two different grades depending on which 
instructor's exam he takes. (2) Some instructors emphasize 
recapitulation of facts rather than applications. Thus, the 
final grade a student receives may reflect his ability to mem- 
orize without regard for his ability to work with new concepts. 
Nonetheless, a student's grade reflects the instructor's judg- 
ment of how well he performs. If some instructors are systema- 
tically biased in their judgments, this has a bearing on their 



2. MF find that the AS measure tends to capture learning changes 
in subject-matter oriented students while grades are a better 
measure for success-oriented students. The teaching skills 
associated with each group tend to differ, suggesting that 
studies of teacher effectiveness should include both mea- 
sures . (15) 

3. This was not the case in the fall quarter where a common 
grading policy was followed to examine the possible bias in- 
troduced by using different grading scales. Our results 
(discussed below) suggest that graduate instructors were 
slightly less generous in the grades they gave students. 
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effectiveness and it is germane to our study. 

The above measures of learning provide a valuable yardstick 
against which the effects of alternative approaches can be eval- 
uated. Nonetheless 5 the principles class affects students in 
other ways worthy of further attention. If a high proportion of 
students plan to take additional courses in economics, for exam- 
ple, this may provide evidence of a successful presentation of 
the material. The same is true if student interest in economics 
increases . 

In summary, the five performance measures are: 
Q-j^: Student score on a modified version of TUCE. 
Q2 : Student score on a twenty question test of economic 

attitudes. 

Qg: Final grade for the quarter (A=4,B=3,C=2,Dor 
below = 1) . 

Q^: End of the quarter interest in economics (5 = very high, 
^ = high, 3 = average, 2 = low, 1 = very low). 

Qg : Willingness to take another course in economics. (1 ^ stu- 
dent intends to take another economics course which is non- 
required, 0 = otherwise). 

Correlation Between Performance Measures 

To illustrate more clearly the point that the various mea- 
sures do not capture the same thing the zero order correlation 
coefficients for each measure are given in Table 1. Note that 
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the largest correlation (between AS and TUCE) is only 
0.28 • Moreover, a consistent pattern emerges; the correlations 
between the AS measure and other variables are generally higher 
than the correlations between TUCE and other measures and be- 
tween final grade and the other measures. Moreover, all of the 
zero order correlations are fairly low. 

TABLE 1 

Zero Order Correlations Among the Dependent Variables 

Final Interest in 
TUCE AS Grade Economics Continuation 

TUCE - 0.28 -.07 -.12 .01 

AS - .20 -.17 .07 

Final Grade - .07 .04 

Interest - .13 

Continuation 



Description of The Data 

At the beginning of the 1972-73 Winter quarter and there- 
after, successive macroeconomic principles classes at the Florida 
State University were asked to complete a detailed questionnaire. 
Information was obtained on student characteristics, grade point 
average, and attitudes on the pre-TUCE and pre-AS exams. A se- 
cond questionnaire was administered at the end of the quarter. 
A total of 612 students completed the course and were tested. 
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Of these, 548 or 90 percent provided usable data. In all, a 
total of 12 classes are included in the sample, taught by one 
Full Professor, two Associates, two Assistants, and three Grad- 
uate Instructors. The means and standard deviations for each of 
the independent variables appear in Appendix Table 1. 

On the basis of prior studies of economic education the 
following variables were selected as the independent variables: 
X3 = Grade point average (5 = 3.5-4.0, 4 = 3.49-3.0,.*., 

1 = 1. 99 or less) 
X2 = Score on TUCE pretest. 
X3 = Score on AS pretest. 

= Sex dummy variable (1 = female, 0 = otherwise) 
Xg = Major dxammy variable (1 = economics or natural 

science, 0 = otherwise) 
Xg = Interest in Economics (5 = very high, 4 = high,..., 
1 = very low) 

Xrj = Class of student (5 = graduate student, 4 = senior, 
. . . , 1 = freshman) 



Two factors account for the response rate. First, some stu- 
dents failed to enter a Social Security number on either the 
pre or post questionnaire. Statistical comparison of the 
unmatched data with the matched data revealed no significant- 
bias from this source. Second, some students either entered 
the course late or failed to show up for the post-test. To 
partially minimize the latter problem, students were told 
that the post-test was a practice exam that vjould be helpful 
in preparing for the final. Since this practice was follow- 
ed in the last two quarters it may have introduced a bias 
into the data. This was not evident in a visual check of 
the class means. 



4. 
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Xg = Continuation dummy variable (1 = student plans to 
take another non-required economics course, 0 = 
otherwise) . 

Y-^ = Number of years since the instructor received his 
Ph,D, (A third year graduate student is two years 
away from receiving his Ph.D. and thus receives a 
-2 in the regression) . 
= Number of hours taught by the instructor during 
the quarter. 

Yg = Graduate Instructor dummy variable (1 = graduate 
instructor, 0 = otherwise). 

Years of Teaching Experience and Student Performance 

Several studies have shown that experienced teachers affect 
student performance. High school students taught by experienced 
teachers receive higher scores on national tests than those 
taught by less experienced teachers. (5) (12) (17) Likewise, ex- 
perienced teachers produce fewer dropouts and more students who 
wish to pursue their education . (23) While the evidence on col- 
lege students enrolled in economics courses is less extensive, 
it appears that experienced teachers improve student performance. 

To explore this hypothesis further, regression analysis was 
utilized to determine the relationship between teacher experience 
and student performance after controlling for variables believed 
to be important. The least squares estimates obtained from OLS 
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estimation appear in Table 2. Each column of the table lists a 
different regression equation; each row gives the regression co- 
efficients for one of the independent variables, along with its 
T-value (in parentheses). Only those coefficients with signifi- 
cant T-values are shown. 

How does teacher experience affect student performance? 
The years beyond Ph.D. coefficient (Y^) is statistically signi- 
ficant in both the TUCE and AS equations but not in the remain- 
ing ones. At the sample mean of 7.6 years of experience, for 
example, the average post-TUCE score is raised 0.3 points or 7 
percent above the mean pre-TUCE score. A similar calculation 
for the post-AS score indicates that the post score rises by 
0*65 points or 6 percent over the pre score mean. These are not 
large increases and they do not reflect well on our existing 
knowledge as to how to teach economic principles. In the post- 
TUCE equation, the effect of one year of additional teacher ex- 
perience (0.039) is substantially less than that of increasing 
the average grade point of students by 0.1 points (0.054). What 
this seems to imply is that student rather than instructor qual- 
ity may be the key to performance on economics exams . This 
would confirm the findings of other researchers regarding stu- 
dent performance in the absence of a faculty member. (20) It al- 
so provides some support of the Coleman Report conclusions at a 
college level. (23) 

Could the findings reported here be due to problems with 
the Y-i variable? Ideally, years of teaching experience rather 



- 9 - 



TABLE 2: 

THE EFFECT OF TEACHER YEARS OF EXPERIENCE 
AS INDICATED BY SEVERAL PERFORMANCE MEASURES 



Independent 
variables 


Post TUCE 


Post AS 


Final 
Grade 


Interest in 
Economics 


Continue on 
in Economici 


Constant term 


5.591 
(16.5) 


12.760 
(18.4) 


3.719 
(14.5) 


1.543 
(8.9) 


0.360 
(6.0) 


Grade Point (X^) 


0.542 
(6.2) 


0.631 
(5.6) 


0.376 
(9.7) 


- 


- 


Pre-TUCE(X2) 


0.219 
(5.5) 










Pre ASCXg) 




0.313 
(9, 5) 


0.050 
(4.3) 


mm 




Sex(X^) 


-0.371 
(2.1) 






-0.306 
(2.8) 


-0.109 
(3.1) 


MajorCXg) 










2/ 


Pre~Interest(Xg 


) 


- 


- 


0.347 
(5.7) 


0.058 
(2.9) 


Class 

Standing(X^) 






-0.1190 
(2.5) 






Pre~Continue(Xg 


) 








0.408 
(8.5) 


Years beyond 


0. 039 
(4.0) 


0.085 
(3.7) 








Number of Hours 
Taught (Y2) 




-0.462 
(5.1) 


1/ 






Adjusted R 


0.18 


0.25 


0.22 


0.08 


0.21 


F* Ratio 


27 .11 


40.94 


33.27 


22 .68 


31.95 


Adjusted 
standard error 


1.87 


2.44 


0.84 


1.17 


0.38 



Table Note: 1/ The coefficient on this variable was significant at 
the 10% level. 



2^/ Significant at 5 percent level .if pre-continue vari- 
able is eliminated but insignificant when it is 
Q included. 
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than years beyond PheD. might have been a better variable to 
use. For the faculty in this sample the distinction is not im- 
portant. It might also have been useful to control for whether 
an instructor previously taught principles or whether his major 
assignment was advanced undergraduate or graduate level courses. 
Again, the distinction is not important in this sample since all 
of the instructors had previous experience teaching principles 
and the within rank assignments were similar. Thus, the vari- 
able used in the study seems to be a reasonable one» 
Consider the other coefficients in Table 2: 

1. Note that the constant term in each of the regression is 
positive and significant r This suggests that other vari- 
ables may be significant which have been excluded from the 
equation. Thusfar, the absence of an effective learning 
theory has prevented researchers from eliminating this prob- 
lem in economic education studies. 

2. The positive coefficient on grade point average suggests 
that students who do well in other courses also do well in 
economics. When this finding is combined with the lack of 
significance for the pre-interest , pre-continue , and student 
major variables it may suggest that a student's learning 
skill, rather than his prior interest in economics, is the 
crucial factor in determining how well he learns economic 
principles. 

3. A student's pre-TUCE score presumably reflects his knowledge 
of economics when he enters the course. Surprisingly, a 
good pre-score does not appear related to a high final grade, 
perhaps again suggesting that prior exposure to the field is 
not as important as the ability to learn. 

4. Performance on the pre-AS exam is significant in determining 
post AS score and final grade. Since the AS test presumably 
measures economic reasoning rather than prior knowledge, 
this may explain why AS is significant and TUCE is not o An 
increase of about 2.5 points on the pre-AS exam adds one 
point to the post AS score and 0.13 points to final grade. 
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5. A student *s sex has no effect on the amount learned. How- 
ever, women are less interested in economics both when they 
enter and when they leave the course. This is also reflec- 
ted in fewer female continuations to other economics courses. 

6. We know surprisingly little about what determines a student's 
interest in economics, either before or after the course .(8) 
(13) (14) This would seem to be a useful area for further 
research. 

7. The hours taught variable is included in the regression to 
control for the effects of variation in workload on 
teacher effectiveness. It is difficult to interpret in a 
small sample, however, since hours taught are not evenly dis- 
tributed across ranks. An increase in the number of hours 
taught appears to have an effect on post AS score and is siff- 
nificant at a 10 percent level in the final grade equation.^ 

8. Finally, although the R^^s are low, the F tests are all sig- 
nificant at a 1 percent level and the W^s are in line with 
those of other researchers in this area. 

- '7 

The Effectiveness of Graduate Instructors 



Other things equal, one might expect that since experienced 
teachers raise student scores on the TUCE and AS exams the addi- 
tion of graduate instructors would reduce student learning. 
Many faculty involved in the principles sequence have noted, 
however, that graduate students enjoy certain advantages: an en- 
thusiasm for the material, a better awareness of what the under- 
graduates don't understand, greater approachability , and perhaps 
a closer rapport with the students. Thus, it is difficult to 



5. The regression coefficient is significant at a 5 percent 
level when all variables are included in the regression. 
A 5 percent level was used in determining which variables 
should be included in the table. 
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predict in advance the effect of assigning graduate students to 

teach the principles course. 

During the summer of 197 3, and in each of the following 

quarters, the economics department assigned graduate students 

to teach in the principles sequence • These students developed 

their material free from departmental directive, although they 

g 

followed a common course outline and used the same textbook. 
The sole criterion for selection of instructors involved their 

•7 

past TA evaluation ratings by students. 

If graduate instructors make a difference, this should be 
indicated by the inclusion of the Y2 variable in the Table 2 re- 
gressions. However, a problem exists • Because of the small 
sample size, and because instructors teach fewer hours, Y2 is 
highly correlated with hours taught. Moreover, since years be- 
yond Ph.D. and the instructor variable are also highly corre- 
lated, the use of the three variables loads the dice against the 
graduate variable. It is, however, useful to know whether the 
Y2 variable has an independent effect, after controlling for 
hours taught and experience. To resolve this dilemma two esti- 
mates are presented. The first excludes hours taught and exper- 
ience from the equations estimated in Table 2 but includes the 



6* The exception to this rule was the fall quarter where one 

faculty section and two graduate instructor sections covered 
the material in a coordinated manner. This was to provide 
further data which will be forthcoming in a future paper. 

7. This practice differs from that of several other departments . (19) 
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Yj variable; the second includes the Y2 variable with all of the 
significant variable equations in Table 2. Since our major in- 
terest is with the graduate instructor and teacher experience 
variables, only the regression coefficients and T-values for 
these are shown below. To facilitate comparison with Table 2, 
adjusted R 'a and F-values are also given. 



TABLE 3 

TOO ESTIMATES OF THE EFFECT OF GRADUATE INSTRUCTORS 
ON STUDENT PERFORMANCE 



Graduate Instructor 


Post 
TUCE 


Post 
AS 


Final 
Grade 


Interest in 
Economics 


Continue on 
in Economics 


Estimate I 


-0.362 
(1.7) 


1.011 
(3.6) 


-0.014 
(0.2) 


0.214 
(1.6) 


0.014 
(0.3) 


Estimate II 


0.193 
(0.7) 


0.024 
(0.1) 


-0. 311 
(1.9) 


0.214 
(1.6) 


0.014 
(0.3) 


Years Beyond Ph.D. 












Estimate II 


0.044 
(3.7) 


0.085 
(3.5) 




* 


ft 


Adiusted R^ 












Estimate I 


0.16 


0.23 


0.21 


0.09 


0.21 


Estimate II 


0.18 


0.25 


0.22 


0.09 


0.21 


F-Ratio 












Estimate I 


23.09 


48.57 


32.76 


16.06 


25.53 


Estimate II 


2-1. 78 


32.68 


27.53 


16.07 


25.53 



Table Note: * denotes that the variable was not included in Table 2 
because it lacked statistical significance. 
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The conclusion suggested by this table is that graduate in- 
structors generally do about as well as faculty in the principles 
course* The specification of the equation makes a difference in 
terms of specific output measures, however. When a straight test 
of graduate instructors as compared to faculty is conducted, the 
results indicate that graduate instructors actually have a posi- 
tive effect on student economic attitudes as compared to the 
faculty- In fact, the regression coefficients suggest that gra- 
duate instructors raise the AS score by an amount about equal to 
that of a faculty member with 12 years of experience. If the 
graduate instructor variable is included in the regression with 
teacher experience and hours taught, however, its regression co- 
efficient is insignificant in the AS equation and negative and 
significant at the 10 percent level in the final grade equation. 
This seems to imply that graduate instructors do about as well 
as faculty on four performance measures but less well on the 
fifth. The two estimates can be reconciled. A micro view of the 
data suggests that the insignificant coefficient on ¥2 in the AS 
equation is due to the fact that the hours taught variable picks 
up the variation captured by estimate I. Eliminating this 

variable from the equation but including teacher experience, we 
again obtain a statistically significant regression coefficient 
for As for the negative coefficient for the final grades 

equation, this appears to have been caused by a difference in 



- 15 



grading procedures rather than by student performance.^ 
Conclusion s 

Prior studies have raised a pcu?adox which appears in our 
findings. Experienced teachers presumably improve student per- 
formance on learning exams yet graduate instructors appear to be 
as effective as faculty in the principles course. Can these two 
findings be explained? What may be involved here are two dif- 
ferent types of teaching skills; in the case of the graduate stu- 
dents, an ability to grasp what students don't understand, a high 
degree of motivation, emd greater approachability ; in the case of 
experienced faculty, a broader background, more experience in 
teaching, greater self-confidence, etc. Given the approach fol- 
lowed in this study, one can only speculate as to which of the 
factors explains our results. Nevertheless, the findings them- 
selves appear to support the view that graduate student instruc- 
tors do as well as the faculty instructors in terms of both the 
learning and interest measures. This conclusion is in accord with 
that of Oates and Quandt for Princeton students . (19) Of course, 
it should be borne in mind that the measures of learning used 
here capture only part of the total learning environment. An 



8. A major difference in grade distribution occurred in the fall 
quarter. A comparison of scores on the common final to final 
grades led to the conclusion that the graduate instructors in 
cur sample were less generous with their grades than were the 
faculty. 
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effective instructor conveys many things not captured by any of 
the measures; command of the subject, caution in accepting unsup- 
ported arguments, a perspective on the economic system, etc We 
have yet to develop measures which provide an effective compari- 
son of faculty and graduate instructors along these lines. 



APPENDIX: TABLE 1 



Means and Standard Deviations for Independent Variables 



Variable Name Mean Standard Deviation 



Attitude Score (Pre) 


11.28 


3.54 


Class Standing 


2 .i|3 


0.85 


Grade Point Average 


2.54 


1.04 


Graduate Instructor 


0.22 


0.41 


Pre-Continue 


0.20 


0.«f«f 


Hours Taught by Instructor 


6.64 


2.74 


Major in Economics 


0.10 


0.37 


Pre -Interest 


2.64 


0.93 


Sex 


0.43 


0.50 


TUCE Score (Pre) 


4.37 


4.44 


Years of Experience of Instructor 


7. SO 


9.21 



o 
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